{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "95656700-f2ba-493b-b3bb-c303f2e83082",
   "metadata": {},
   "source": [
    "# Interesting Findings\n",
    "### Clouds\n",
    "Clouds obscure land features and are often interpreted as buildings.\n",
    "\n",
    "### Roof solar panels\n",
    "Solar panels on roofs often appear as damaged buildings.\n",
    "\n",
    "_Tip:  Include more cloud images in the training dataset to improve building and damage detection._"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "43066c00-4305-47a6-90e1-2bad2db62136",
   "metadata": {
    "tags": []
   },
   "source": [
    "# File Formats and Datasets"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "03a78a40-f47c-4d76-9dad-585aa38fe74e",
   "metadata": {
    "tags": []
   },
   "source": [
    "### [GeoJSON](https://geojson.org/)\n",
    "Geographic JSON format with support for multi-polygons and custom properties.\n",
    "\n",
    "```\n",
    "{\n",
    "  \"type\": \"FeatureCollection\",\n",
    "  \"crs\": {\n",
    "      \"type\": \"name\", \n",
    "      \"properties\": { \n",
    "          \"name\": \"urn:ogc:def:crs:OGC:1.3:CRS84\" \n",
    "      } \n",
    "  },\n",
    "  \"features\": \n",
    "     [\n",
    "       {\n",
    "         \"type\": \"Feature\", \n",
    "         \"properties\": {\n",
    "             \"DAMAGE_AREA_ID\": \"{f8432ec2-49d5-42c8-8065-4c8d8b54e773}\", \n",
    "             \"CREATED_DATE\": \"2020-10-30T21:17:14+00:00\", \n",
    "             \"MODIFIED_DATE\": \"2020-11-19T20:03:11+00:00\", \n",
    "             \"FEATURE_USER_ID\": \"smckelvey_partner\", \n",
    "             \"OBSERVATION_USER_ID\": \"smckelvey_partner\", \n",
    "             \"REGION_ID\": 5, \n",
    "             \"LABEL\": \"RF-1\", \n",
    "             \"HOST_CODE\": 20.0, \n",
    "             \"HOST\": \"California red fir\", \n",
    "             \"HOST_GROUP_CODE\": null, \n",
    "             \"HOST_GROUP\": null, \n",
    "             \"DCA_CODE\": 11050, \n",
    "             \"DCA_COMMON_NAME\": \"fir engraver\", \n",
    "             \"DAMAGE_TYPE_CODE\": 2, \n",
    "             \"DAMAGE_TYPE\": \"Mortality\", \n",
    "             \"PERCENT_AFFECTED_CODE\": 1, \n",
    "             \"PERCENT_AFFECTED\": \"Very Light (1-3%)\", \n",
    "             \"PERCENT_MIN\": 1.0, \n",
    "             \"PERCENT_MAX\": 3.0, \n",
    "             \"PERCENT_MID\": 2.0, \n",
    "             \"NOTES\": \"NA20; 20200730; 035H\", \n",
    "             \"OBSERVATION_COUNT\": \"SINGLE\", \n",
    "             \"COLLECTION_MODE\": \"DesktopGIS\", \n",
    "             \"SNAPGRID_ROW\": null, \n",
    "             \"SNAPGRID_COLUMN\": null, \n",
    "             \"AREA_TYPE\": \"POLYGON\", \n",
    "             \"ACRES\": 5.09, \n",
    "             \"STATUS\": 1, \n",
    "             \"SURVEY_YEAR\": 2020, \n",
    "             \"LEGACY_SEVERITY_CODE\": null, \n",
    "             \"LEGACY_SEVERITY\": null, \n",
    "             \"LEGACY_PATTERN_CODE\": null, \n",
    "             \"LEGACY_PATTERN\": null, \n",
    "             \"LEGACY_TPA\": null, \n",
    "             \"LEGACY_NO_TREES\": null, \n",
    "             \"LEGACY_FOREST_TYPE_CODE\": null, \n",
    "             \"LEGACY_FOREST_TYPE\": null, \n",
    "             \"GRP\": null, \n",
    "             \"OBSERVATION_ID\": \"{333ed692-0a37-4990-a368-4d503b791621}\", \n",
    "             \"IDS_DATA_SOURCE\": \"91\", \n",
    "             \"DATA_SOURCE_NAME\": \"REMOTE_SENSING\", \n",
    "             \"US_AREA\": \"CONUS\", \n",
    "             \"SHAPE_Length\": 595.21228600142638, \n",
    "             \"SHAPE_Area\": 20579.801704105612 \n",
    "         }, \n",
    "         \"geometry\": { \n",
    "             \"type\": \"MultiPolygon\", \n",
    "             \"coordinates\": [ \n",
    "                 [ \n",
    "                     [ \n",
    "                         [ -120.20806708986845, 38.604999600002188 ], \n",
    "                         [ -120.210326122885249, 38.605113969931573 ], \n",
    "                         [ -120.210303609363322, 38.605562702761141 ], \n",
    "                         [ -120.208604606082048, 38.606291053520479 ], \n",
    "                         [ -120.207699241299181, 38.605333861676037 ], \n",
    "                         [ -120.20806708986845, 38.604999600002188 ] \n",
    "                     ] \n",
    "                 ] \n",
    "             ] \n",
    "         } \n",
    "     }\n",
    "   ]\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "73919730-ecd7-49e5-9664-fa9d055beb34",
   "metadata": {
    "tags": []
   },
   "source": [
    "### [GeoTIFF](https://en.wikipedia.org/wiki/GeoTIFF)\n",
    "\n",
    "Geographic Tagged Image File Format (GeoTIFF) organizes and stores geospatial raster image* datasets such as satellite imagery and terrain models.  \n",
    "\n",
    "GeoTIFF stores spatial information and coordinates within the image file including the raster’s cell size and coordinate reference system (CRS).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bb9273be-71bb-49ef-9151-891e5daa97b0",
   "metadata": {},
   "source": [
    "### [Cloud-Optimized GeoTIFF (COG)](https://www.cogeo.org/)\n",
    "\n",
    "A Cloud Optimized GeoTIFF (COG) is a regular GeoTIFF file with pixels organized in a way that enables `HTTP GET` range requests.  This allows streaming the GeoTIFF as needed versus having to download the whole file."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6c1aaddf-6669-4be5-a57a-214c9739e6a0",
   "metadata": {},
   "source": [
    "### [AWS Open Data Registry - Disaster Response](https://registry.opendata.aws/tag/disaster-response/)\n",
    "This registry exists to help people discover and share datasets that are available via AWS resources."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f847dd2-149f-492f-b544-8429c40594b9",
   "metadata": {
    "tags": []
   },
   "source": [
    "### [Forestry Insect-and-Disease Detection Survey](https://www.fs.fed.us/foresthealth/applied-sciences/mapping-reporting/detection-surveys.shtml)\n",
    "\n",
    "Collecting data on the health of trees in areas affected by insects and diseases throughout U.S. forests."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e6523ff4-1fdc-45c8-9c53-97137bbdc796",
   "metadata": {
    "tags": []
   },
   "source": [
    "### [Landsat Satellite Imagery 30m](https://earthexplorer.usgs.gov/)\n",
    "\n",
    "Space-based images of the Earth’s land surface at 30m resolution.  These images serve as valuable resources for land-use and land-change research over time."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9ffe102f-6c48-432a-8fb8-ebcb0e10369b",
   "metadata": {},
   "source": [
    "### [Copernicus Digital Elevation Model (DEM)](https://registry.opendata.aws/copernicus-dem/)\n",
    "Multi-purpose dataset that represents the Earth's surface including buildings, infrastructure, and vegetation."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e22f7820-73fc-4b04-a1e1-0dffc0f4ea59",
   "metadata": {},
   "source": [
    "### [Moderate Resolution Imaging Spectroradiometer (MODIS) 500m](https://registry.opendata.aws/modis/)\n",
    "\n",
    "Views the entire surface of the Earth every 1-2 days to describe land, oceans, and atmosphere at 500m resolution.  This dataset is used to study trends across the global."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f505436e-ab39-432a-bbb1-098494c7a2ae",
   "metadata": {},
   "source": [
    "### [European Space Agency (ESA) WorldCover 10m](https://registry.opendata.aws/esa-worldcover/)\n",
    "European Space Agency's global land cover map at 10m resolution using 3x3 tiles across the following 11 categories:\n",
    "* Tree cover \n",
    "* Shrublan\n",
    "* Grassland\n",
    "* Croplan\n",
    "* Built-up\n",
    "* Bare/sparse vegetation\n",
    "* Snow and ice\n",
    "* Permanent water bodies\n",
    "* Herbaceous wetland\n",
    "* Mangroves\n",
    "* Moss and lichen"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3329db18-1025-4329-9472-463023e7451a",
   "metadata": {},
   "source": [
    "### [ECMWF ERA5 Reanalysis](https://registry.opendata.aws/ecmwf-era5/)\n",
    "Provides essential atmospheric meteorological characteristics for the following:\n",
    "* air temperature\n",
    "* pressure and wind at different altitudes\n",
    "* rainfall\n",
    "* soil moisture content\n",
    "* sea-surface temperature\n",
    "* wave height"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2ccab192-c225-4910-8897-872edbcd963d",
   "metadata": {
    "tags": []
   },
   "source": [
    "### [Fire Information for Resource Management System (FIRMS)](https://firms.modaps.eosdis.nasa.gov/)\n",
    "Near-realtime monitoring and detection of active fires and thermal oddities within 3 hours of satellite observation.  Supports active-fire alerts as well as history thermal data over time."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7e814246-c728-44ac-a568-d636f9c2149a",
   "metadata": {},
   "source": [
    "### [SpaceNet2 Building Detection]()\n",
    "Provides labeled data for roads, building footprints, and other points of interest in the following number of buildings across 4 cities:\n",
    "* 151k buildings and 216 sq km in Las Vegas\n",
    "* 23k buildings and 1,030 sq km in Paris\n",
    "* 92K buildings and 1,000 sq km in Shanghai\n",
    "* 35K buildings and 765 sq km in Khartoum"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0e930dce-3092-4945-a2c7-ee3e3a5be736",
   "metadata": {},
   "source": [
    "### [OpenStreetMap](https://www.openstreetmap.org/#map=4/38.01/-95.84)\n",
    "Collaborative, crowd-sourced geographic database of the world from over 2 million registered users using GPS devices, aerial photography, and local knowledge of the area."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d6dd758d-7c62-407e-ba76-54adaf804e46",
   "metadata": {
    "tags": []
   },
   "source": [
    "### [xBD Dataset](https://xview2.org/dataset)\n",
    "Annotated, high-resolution satellite imagery for building-damage assessment.\n",
    "\n",
    "* 850,000+ buildings including 4 levels of damage:  none, minor, major, destroyed\n",
    "* 3,700+ pairs of pre and post disaster satellite images\n",
    "* 6 disaster types including earthquake, fire, flooding, tsunami, volcano, wind\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "815af904-fa2d-43aa-a0be-543a57d93b6a",
   "metadata": {
    "tags": []
   },
   "source": [
    "### [Maxar Open Data Program](https://www.maxar.com/open-data)\n",
    "Open datasets for sudden-onset major crisis events such as the following:\n",
    "* Hurricane Michael Oct 7-16, 2018\n",
    "* Hurricane Florence Sep 10-19, 2018\n",
    "* Hurricane Harvey Aug 17 - Sep 2, 2017\n",
    "* Hurricane Matthew Sep 28 - Oct 10, 2016\n",
    "* Monsoon in Nepal, India, Bangladesh Jul - Sep, 2017\n",
    "* Moore, OK Tornado May 20, 2013\n",
    "* Tuscaloosa, AL Tornado Apr 27, 2011\n",
    "* Joplin, MO Tornado May 22, 2011"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "249a9771-0e2a-4c1a-9cb4-c4791ee139af",
   "metadata": {},
   "source": [
    "### [Sentinel2](https://sentinel.esa.int/web/sentinel/missions/sentinel-2)\n",
    "\n",
    "Satellites that monitor variability in Earth's surface conditions to help study the following:\n",
    "* Climate change \n",
    "* Emergency management\n",
    "* Security\n",
    "* Border protection\n",
    "* Maritime surveillance"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8c68c566-d50c-496e-89e2-484df60b5772",
   "metadata": {
    "tags": []
   },
   "source": [
    "### [ImageNet](https://www.image-net.org)\n",
    "ImageNet contains more than 14 million images across 20,000 categories and includes over 1 million images with bounding boxes."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "830b1fad-ba18-489f-8755-2b43abb793e1",
   "metadata": {
    "tags": []
   },
   "source": [
    "# Models and Libraries"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c8d1c36-fabc-4127-9b62-459b948354e4",
   "metadata": {
    "tags": []
   },
   "source": [
    "### [ResUNet](https://arxiv.org/abs/1711.10684)\n",
    "\n",
    "Semantic segmentation neural network of roads within aerial images used for disaster-response research."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "093d81a2-3004-43fc-bfb9-3193a8fd029d",
   "metadata": {},
   "source": [
    "### [GeoPandas](https://github.com/geopandas)\n",
    "\n",
    "Adds support for geographic data to Pandas.  Implements `GeoSeries` and `GeoDataFrame` as subclasses of `pandas.Series` and `pandas.DataFrame`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a11cc116-26ca-4146-a606-feee686f3477",
   "metadata": {
    "tags": []
   },
   "source": [
    "### [EarthPy](https://earthpy.readthedocs.io/) \n",
    "Python package devoted to working with spatial-raster and remote-sensing vector data."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7ea06ed1-5fe9-42f5-bc26-f94818214a41",
   "metadata": {},
   "source": [
    "### [Rasterio](https://rasterio.readthedocs.io/)\n",
    "Reads and writes GeoTIFF files and provides a Python API based on Numpy N-dimensional arrays."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6f7857f-4cee-462a-82ea-c086b5f44328",
   "metadata": {},
   "source": [
    "### [Scikit-Learn](https://scikit-learn.org/stable/tutorial/machine_learning_map/index.html)\n",
    "* K-Nearest Neighbor (KNN)\n",
    "* PCA\n",
    "* SVD\n",
    "* TSNE\n",
    "* Binary Classification\n",
    "* Multi-class Classification\n",
    "* Logistic Regression\n",
    "* Decision Tree\n",
    "* Random Forest"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "56bab6aa-a554-4e7c-915c-7021e740a2e5",
   "metadata": {
    "tags": []
   },
   "source": [
    "### [Numba](https://numba.pydata.org/)\n",
    "\"Numpy for GPU\" using Nvidia CUDA library"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1e06f300-a25b-4585-b691-8eca427726bb",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "reMars2022:Python",
   "language": "python",
   "name": "conda-env-reMars2022-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
